Various interfaces are known for interconnecting host computer systems to storage systems, such as disk drive mass storage systems. Particular interfaces involve respective particular data transfer protocols and system interconnections. The IBM Enterprise System Connection Architecture (ESCON) is a flexible interface or interconnection environment that is used to move data from a host source to storage and back. ESCON, as it is known in the art, combines technology and architecture to include fiber optic cabling for transmission and reception of data. ESCON implements dynamic connectivity through switched point to point topology and data flow interconnectivity with other networks. The ESCON I/O architecture exploits the fiber optic technology implementation and the concept of dynamic connectivity in providing a flexible, extensible interface.
The ESCON architecture has become a widely accepted standard for data communications in large scale computer environments, replacing the more traditional IBM “Bus and Tag” protocol. ESCON employs serially encoded data transmission techniques in place of parallel data protocols.
The ESCON architecture provides inherent advantages such as: information transfer using fiber optic technology; higher rates of data transfer (17MBytes/sec); extended distances (3 km typical with extension to 60 km); dynamic connectivity through switched point-to-point data flow; and interconnectivity with local and wide area networks
Application of ESCON as the means for data communications can, in addition to the asserted enhancements in data throughput rates and distance, provide other meaningful benefits such as: greater connection flexibility of equipment; reduced floor loading due to significant cable weight and size reductions; customer configuration expansion or reconfiguration with minimal or no disruption; increased data integrity and security; and reduced cost of ownership through more effective utilization of equipment.
The ESCON architecture has been adopted by many other manufacturers of computer equipment as a basic input/output protocol and it has been accepted as an ANSI Standard (“SBCON”). The technical details of the IBM ESCON interface are described, among other places, in various IBM publications including INTRODUCING ENTERPRISE SYSTEMS CONNECTION, IBM 3990 ESCON FUNCTION, INSTALLATION AND MIGRATION, IBM 3990 STORAGE CONTROL ESCON FEATURES PRESENTATION GUIDE, ENTERPRISE SYSTEMS ARCHITECTURE/390 ESCON I/O INTERFACE AND ENTERPRISE SYSTEMS ARCHITECTURE, which are incorporated herein by reference.
The various devices interconnected to a host system with ESCON I/O include storage systems known as “Integrated Cached Disk Arrays,” (“ICDAs”), which are typically an array of small inexpensive disk drives integrated into a single chassis. High speed caches are implemented between the host and disks in ICDAs to yield improved performance. One family of known ICDA products, known as SYMMETRIX produced by EMC Corporation, Hopkinton, Mass., provides a high reliability array of drives, and offers great flexibility in terms of performance enhancements such as: mirroring; greater data availability; greater data transfer rates over distributed buses; and various levels of redundancy implemented in systems referred to as “RAID systems” (“Redundant Arrays of Inexpensive Disks”).
The EMC2 Symmetrix architecture, generally illustrated in FIG. 1, integrates a high speed cache or global memory between a disk array and a host computer or CPU. The functional elements generally required to integrate the cache include a host-to-cache interface (which in one implementation is an IBM standard ESCON interface referred to as a host “ESCON Adapter”—EA), and a cache-to-disk drives interface (which may be a Small Computer Systems Interface, “SCSI”, referred to as a “Disk Adapter”—DA). The EA and DA interface boards are generically referred to as “Directors”. The Symmetrix architecture operates under a “cache all” policy, meaning that all transfers, i.e. from the host to the drives or from the drives to the host, go through cache. The principal function of the Directors is the movement of data between the host and Global Memory (cache) or between Global Memory and the Disk Drives.
The Global Memory Bus (GMB), between the EA and cache and between the DA and cache in Symmetrix, actually consists of two portions or identical buses designated “A” and “B”. The use of two buses improves performance and eliminates a possible single point of failure. Each bus has independent arbitration and consists of a 32 bit address bus plus 1 parity, a 64 bit data bus with 8 bits of Error Correction Code (ECC) check bits and a number of control lines with parity. The smallest data transfer that may take place over the GMB is 64 bits during each access (however, byte, word and longword operations are performed within a Director). The SYMMETRIX family of ICDAs are described in detail in the Symmetrix Product Manuals (for Models 5500, 52XX, 5100, 3500, 32XX and 3100) which are incorporated herein by reference.
The ESCON adapter interface(s) and disk adapter interface(s), i.e. the bus(es) between the host and EAs and between the disk array and DAs respectively, are 8 bit and in some cases 16 bit interfaces in Symmetrix. Thus the bytes received from a host have to be assembled into a 64-bit memory word for transfer to Global Memory, i.e. cache. Similarly, 64-bit memory words from Global Memory have to be disassembled into bytes for transmission over the interface(s).
Assembly/disassembly of data to/from Global Memory is carried out by plural gate arrays located on each Director.
The Symmetrix family of ICDAs is designed around a pipelined architecture. A pipeline or pipe in the system is a registered path along which data is clocked to move it from one location to another.
The Directors, generally, are designed around a common pipelined architecture moving data along the “pipe” or pipeline under microprocessor control. It is the pipelines that move the data when the system is operating, not the controlling microprocessors. The microprocessors set-up the pipes to perform a transfer and monitor the pipelines for errors during the transfer. The Directors in known Symmetrix systems incorporate a dual control processor architecture including a first processor referred to as “X” and a second processor referred to as “Y”. The dual processor architecture is configured to share substantial resources in order to keep hardware requirements to a minimum. Each control processor in a “front end” Director, i.e. transferring data to/from Global Memory, is typically responsible for two pipelines designated “A” and “B” and respective Direct Multiple Access (DMA) and Direct Single Access (DSA) pipelines, for Global Memory access.
Known Symmetrix systems can be configured with ESCON adapter circuitry for communication with hosts having ESCON I/O, as disclosed in U.S. patent application Ser. No. 08/753,673 which is incorporated herein by reference. FIG. 2 illustrates a known ESCON front end Adapter or Director, configured for transferring data between a host ESCON I/O (not shown) according to the IBM Bus ESCON standard known in the art and the ICDA incorporating the Global Memory. Data in accordance with the ESCON protocol is received on the ESCON Director by ESCON Receiver/Transmitter Gate Arrays (Rx/Tx GA) which receive and transmit ESCON data between the host ESCON I/O and the ESCON Director pipes in accord with the ESCON I/O protocol. ESCON data is transmitted in two physical pipes that are configured to operate as four pipes, two on each side: an A and B pipe (XA and XB for transmit and receive, respectively) on the X side controlled by an X control processor, and an A and B pipe (YA and YB for transmit and receive, respectively) on the Y side controlled by a Y control processor. Parallel ESCON data is passed in the pipes on the ESCON front end Director on a bidirectional bus connected to respective ESCON Receiver/Transmitter Gate Arrays (Rx/Tx GA), which in turn assemble/disassemble 64 bit memory words. Error detection and correction is performed in the ESCON pipe using a pass through EDAC device. Dual Port Ram (DPR) is implemented in the pipe for buffering data transferred between the ESCON host (not shown) and Global Memory.
Each processor on the ESCON Director has its own Memory Data Register (MDR) to support DMA/DSA activity. The MDR provides a 72 bit wide register set, 64 bit data and 8 bits parity, comprised of upper and lower words that can be independently read or written by the respective microprocessor. Similarly, the MDR performs the assembly and disassembly of 64 bit Global Memory words.
The two microprocessors on the known ESCON Director perform functions associated with managing data flow to and from Global Memory, and additionally are accessible as individual points of contact for a user to configure memory and/or test and exercise the memory and pipelines (e.g. for diagnostic purposes). Such an implementation can disadvantageously interfere with the data flow management and adds complexity to the system. Complexity is inherent in the configuration(s) known in the art, in that the individual pipeline or line processor's activities, beyond data flow management, must be coordinated with other line processors' activities. Furthermore, involving line processors in other than data flow management leads to bottlenecks on the backplane bus that negatively affect system performance. For example, inter-processor communication, i.e. communication between line processors, interferes with the system's data flow capabilities.
Known ICDA implementations, such as known Symmetrix systems, are designed according to a two level architecture. That is, the Director interface is a first level interface that works in conjunction with a second level adapter board that is directly connected to the host ESCON I/O. The ESCON Receiver/Transmitter Gate Arrays (Rx/Tx GA) on the Director transmit data to or receive data from the second level ESCON adapter board that effects data conversion using devices known in the art. The second level adapter board converts parallel data to serial ESCON data using a Cypress CY7B923 device (for transmission to the host), or converts serial ESCON data to parallel with a Cypress CY7B933 (for receipt by the ESCON Director). A processor on the second level adapter board handles protocol operations and basically translates ESCON information into a generic interface protocol, or interchange format. The adapter board includes memory associated with the adapter processor, and all of the data from the ESCON protocol engines is stored temporarily in this memory. The processor on the adapter then has to figure out by looking through the stored data where header information is and where the data is. The adapter processor then writes in the adapter memory information about where the data starts and ends. In addition, the processor interprets commands from the host and writes into the adapter memory what the command is and what the ESCON command means in a generic fashion.
The Director card processor then finds this information. The processor on the Director(s) polls through the adapter memory space to find directives from the adapter. For example, an ESCON Director polling the adapter memory could find a write operation along with an indication of where the data to be written is located. The converted ESCON information found can then be handled by the processor(s) on the Director(s), which take the data and actually do transfers to/from the Global Memory.
Such a two level architecture results in significant loss of performance, in part because of the polling for the data transfer information. With the two levels, the adapter card at one level has to figure out what the command is, where the data is, translate it to the interchange format and store all this in the adapter memory. At the second level, the Director polls and retrieves the interchange information. There is very little pipelining that can be effected in such an implementation. Inefficiencies result from the architecture wherein operations are completed at one level and handed off to another level. Parallelism is not exploited.
Similarly, known ICDA implementations, and other arrays of inexpensive disks that transfer data according to the known ESCON protocol, receive user data for disk storage in the context of a frame that includes header information and frame information. Known integrated cache systems typically store the data in the cache memory along with the header and frame information. Such an implementation is not an efficient use of the cache memory capacity. Disadvantageously, complex software has to be configured to effectively track the location of the data in memory among ESCON header/frame information. Movement of cached data, i.e. going to or from the disk array, in known systems requires that the data be discriminated from extraneous cached protocol information which negatively affects transmission speed and efficiency.
Additionally, storage of user data among header/frame information increases the chances of conflicts arising between line processors accessing the cache to retrieve user data and other line processors perusing the cache to ascertain pertinent header/frame information. Complex pointer management is required to maintain coherency between multiple processors accessing cache storing data and header/frame information.